{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FINN - Functional Verification of End-to-End Flow\n",
    "-----------------------------------------------------------------\n",
    "\n",
    "**Important: This notebook depends on the tfc_end2end_example notebook, because we are using models that are available at intermediate steps in the end-to-end flow. So please make sure the needed .onnx files are generated to run this notebook.**\n",
    "\n",
    "In this notebook, we will show how to take the intermediate results of the end-to-end tfc example and verify their functionality with different methods. In the following picture you can see the section in the end-to-end flow about the *Simulation & Emulation Flows*. Besides the methods in this notebook, there is another one that is covered in the Jupyter notebook [tfc_end2end_example](tfc_end2end_example.ipynb): remote execution. The remote execution allows functional verification directly on the PYNQ board, for details please have a look at the mentioned Jupyter notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"verification.png\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use the following helper functions, `showSrc` to show source code of FINN library calls and `showInNetron` to show the ONNX model at the current transformation step. The Netron displays are interactive, but they only work when running the notebook actively and not on GitHub (i.e. if you are viewing this on GitHub you'll only see blank squares)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from finn.util.basic import make_build_dir\n",
    "from finn.util.visualization import showSrc, showInNetron\n",
    "import os\n",
    "\n",
    "build_dir = os.environ[\"FINN_BUILD_DIR\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To verify the simulations, a \"golden\" output is calculated as a reference. This is calculated directly from the Brevitas model using PyTorch, by running some example data from the MNIST dataset through the trained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pkgutil import get_data\n",
    "import onnx\n",
    "import onnx.numpy_helper as nph\n",
    "import torch\n",
    "from finn.util.test import get_test_model_trained\n",
    "\n",
    "fc = get_test_model_trained(\"TFC\", 1, 1)\n",
    "raw_i = get_data(\"qonnx.data\", \"onnx/mnist-conv/test_data_set_0/input_0.pb\")\n",
    "input_tensor = onnx.load_tensor_from_string(raw_i)\n",
    "input_brevitas = torch.from_numpy(nph.to_array(input_tensor).copy()).float()\n",
    "output_golden = fc.forward(input_brevitas).detach().numpy()\n",
    "output_golden"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulation using Python <a id='simpy'></a>\n",
    "\n",
    "If an ONNX model consists of [standard ONNX](https://github.com/onnx/onnx/blob/main/docs/Operators.md) nodes and/or FINN custom operations that do not belong to the fpgadataflow (`backend` $\\neq$ `fpgadataflow`) this model can be checked for functionality using Python.\n",
    "\n",
    "To simulate a standard ONNX node [onnxruntime](https://github.com/microsoft/onnxruntime) is used. onnxruntime is an open source tool developed by Microsoft to run standard ONNX nodes. For the FINN custom op nodes execution, functions are defined. The following is an example of the execution function of a XNOR popcount node.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from qonnx.custom_op.general.xnorpopcount import xnorpopcountmatmul\n",
    "showSrc(xnorpopcountmatmul)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function contains a description of the behaviour in Python and can thus calculate the result of the node.\n",
    "\n",
    "This execution function and onnxruntime is used when `execute_onnx` from `onnx_exec` is applied to the model. The model is then simulated node by node and the result is stored in a context dictionary, which contains the values of each tensor at the end of the execution. To get the result, only the output tensor has to be extracted.\n",
    "\n",
    "The procedure is shown below. We take the model right before the nodes should be converted into HLS layers and generate an input tensor to pass to the execution function. The input tensor is generated from the Brevitas example inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from qonnx.core.modelwrapper import ModelWrapper\n",
    "input_dict = {\"global_in\": nph.to_array(input_tensor)}\n",
    "\n",
    "model_for_sim = ModelWrapper(build_dir+\"/tfc_w1a1_ready_for_hls_conversion.onnx\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import finn.core.onnx_exec as oxe\n",
    "output_dict = oxe.execute_onnx(model_for_sim, input_dict, return_full_exec_context=False)\n",
    "output_pysim = output_dict[list(output_dict.keys())[0]]\n",
    "\n",
    "\n",
    "\n",
    "if np.isclose(output_pysim, np.where(output_golden[0]==np.amax(output_golden[0])), atol=1e-3).all():\n",
    "    print(\"Results are the same!\")\n",
    "else:\n",
    "    print(\"The results are not the same!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result is compared with the theoretical \"golden\" value for verification."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulation (cppsim) using C++\n",
    "\n",
    "When dealing with HLS custom op nodes in FINN the simulation using Python is no longer sufficient. After the nodes have been converted to HLS layers, the simulation using C++ can be used. To do this, the input tensor is stored in a .npy file and C++ code is generated that reads the values from the .npy array, streams them to the corresponding finn-hlslib function and writes the result to a new .npy file. This in turn can be read in Python and processed in the FINN flow. For this example the model after setting the folding factors in the HLS layers is used, please be aware that this is not the full model, but the dataflow partition, so before executing at the end of this section we have to integrate the model back into the parent model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_for_cppsim = ModelWrapper(build_dir+\"/tfc_w1_a1_set_folding_factors.onnx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To generate the code for this simulation and to generate the executable two transformations are used:\n",
    "* `PrepareCppSim` which generates the C++ code for the corresponding hls layer\n",
    "* `CompileCppSim` which compules the C++ code and stores the path to the executable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from finn.transformation.fpgadataflow.prepare_cppsim import PrepareCppSim\n",
    "from finn.transformation.fpgadataflow.compile_cppsim import CompileCppSim\n",
    "from qonnx.transformation.general import GiveUniqueNodeNames\n",
    "\n",
    "model_for_cppsim = model_for_cppsim.transform(GiveUniqueNodeNames())\n",
    "model_for_cppsim = model_for_cppsim.transform(PrepareCppSim())\n",
    "model_for_cppsim = model_for_cppsim.transform(CompileCppSim())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we take a look at the model using netron, we can see that the transformations introduced new attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_for_cppsim.save(build_dir+\"/tfc_w1_a1_for_cppsim.onnx\")\n",
    "showInNetron(build_dir+\"/tfc_w1_a1_for_cppsim.onnx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following node attributes have been added:\n",
    "* `code_gen_dir_cppsim` indicates the directory where the files for the simulation using C++ are stored\n",
    "* `executable_path` specifies the path to the executable\n",
    "\n",
    "We take now a closer look into the files that were generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from qonnx.custom_op.registry import getCustomOp\n",
    "\n",
    "fc0 = model_for_cppsim.graph.node[1]\n",
    "fc0w = getCustomOp(fc0)\n",
    "code_gen_dir = fc0w.get_nodeattr(\"code_gen_dir_cppsim\")\n",
    "!ls {code_gen_dir}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides the .cpp file, the folder contains .h files with the weights and thresholds. The shell script contains the compile command and *node_model* is the executable generated by compilation. Comparing this with the `executable_path` node attribute, it can be seen that it specifies exactly the path to *node_model*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To simulate the model the execution mode(exec_mode) must be set to \"cppsim\". This is done using the transformation SetExecMode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from finn.transformation.fpgadataflow.set_exec_mode import SetExecMode\n",
    "\n",
    "model_for_cppsim = model_for_cppsim.transform(SetExecMode(\"cppsim\"))\n",
    "model_for_cppsim.save(build_dir+\"/tfc_w1_a1_for_cppsim.onnx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before the model can be executed using `execute_onnx`, we integrate the child model in the parent model. The function reads then the `exec_mode` and writes the input into the correct directory in a .npy file. To be able to read this in C++, there is an additional .hpp file ([npy2apintstream.hpp](https://github.com/Xilinx/finn/blob/main/src/finn/qnn-data/cpp/npy2apintstream.hpp)) in FINN, which uses cnpy to read .npy files and convert them into streams, or to read a stream and write it into an .npy. [cnpy](https://github.com/rogersce/cnpy) is a helper to read and write .npy and .npz formates in C++.\n",
    "\n",
    "The result is again compared to the \"golden\" output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parent_model = ModelWrapper(build_dir+\"/tfc_w1_a1_dataflow_parent.onnx\")\n",
    "sdp_node = parent_model.graph.node[1]\n",
    "child_model = build_dir + \"/tfc_w1_a1_for_cppsim.onnx\"\n",
    "getCustomOp(sdp_node).set_nodeattr(\"model\", child_model)\n",
    "output_dict = oxe.execute_onnx(parent_model, input_dict)\n",
    "output_cppsim = output_dict[list(output_dict.keys())[0]]\n",
    "\n",
    "if np.isclose(output_cppsim, np.where(output_golden[0]==np.amax(output_golden[0])), atol=1e-3).all():\n",
    "    print(\"Results are the same!\")\n",
    "else:\n",
    "    print(\"The results are not the same!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Emulation (rtlsim) using PyVerilator\n",
    "\n",
    "The emulation using [PyVerilator](https://github.com/maltanar/pyverilator) can be done after IP blocks are generated from the corresponding HLS layers. Pyverilator is a tool which makes it possible to simulate verilog files using verilator via a python interface.\n",
    "\n",
    "We have two ways to use rtlsim, one is to run the model node-by-node as with the simulation methods, but if the model is in the form of the dataflow partition, the part of the graph that consist of only HLS nodes could also be executed as whole."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because at the point where we want to grab and verify the model, the model is already in split form (parent graph consisting of non-hls layers and child graph consisting only of hls layers) we first have to reference the child graph within the parent graph. This is done using the node attribute `model` for the `StreamingDataflowPartition` node.\n",
    "\n",
    "First the procedure is shown, if the child graph has ip blocks corresponding to the individual layers, then the procedure is shown, if the child graph already has a stitched IP."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Emulation of model node-by-node\n",
    "\n",
    "The child model is loaded and the `exec_mode` for each node is set. To prepare the node-by-node emulation the transformation `PrepareRTLSim` is applied to the child model. With this transformation the emulation files are created for each node and can be used directly when calling `execute_onnx()`. Each node has a new node attribute \"rtlsim_so\" after transformation, which contains the path to the corresponding emulation files. Then it is saved in a new .onnx file so that the changed model can be referenced in the parent model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from finn.transformation.fpgadataflow.prepare_rtlsim import PrepareRTLSim\n",
    "from finn.transformation.fpgadataflow.prepare_ip import PrepareIP\n",
    "from finn.transformation.fpgadataflow.hlssynth_ip import HLSSynthIP\n",
    "\n",
    "test_fpga_part = \"xc7z020clg400-1\"\n",
    "target_clk_ns = 10\n",
    "\n",
    "child_model = ModelWrapper(build_dir + \"/tfc_w1_a1_set_folding_factors.onnx\")\n",
    "child_model = child_model.transform(GiveUniqueNodeNames())\n",
    "child_model = child_model.transform(PrepareIP(test_fpga_part, target_clk_ns))\n",
    "child_model = child_model.transform(HLSSynthIP())\n",
    "child_model = child_model.transform(SetExecMode(\"rtlsim\"))\n",
    "child_model = child_model.transform(PrepareRTLSim())\n",
    "child_model.save(build_dir + \"/tfc_w1_a1_dataflow_child.onnx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to load the parent model and set the node attribute `model` in the StreamingDataflowPartition node (`sdp_node`). Afterwards the `exec_mode` is set in the parent model in each node and the model can be executed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# parent model\n",
    "model_for_rtlsim = ModelWrapper(build_dir + \"/tfc_w1_a1_dataflow_parent.onnx\")\n",
    "# reference child model\n",
    "sdp_node = getCustomOp(model_for_rtlsim.graph.node[1])\n",
    "sdp_node.set_nodeattr(\"model\", build_dir + \"/tfc_w1_a1_dataflow_child.onnx\")\n",
    "\n",
    "model_for_rtlsim = model_for_rtlsim.transform(SetExecMode(\"rtlsim\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_dict = oxe.execute_onnx(model_for_rtlsim, input_dict)\n",
    "output_rtlsim = output_dict[list(output_dict.keys())[0]]\n",
    "\n",
    "if np.isclose(output_rtlsim, np.where(output_golden[0]==np.amax(output_golden[0])), atol=1e-3).all():\n",
    "    print(\"Results are the same!\")\n",
    "else:\n",
    "    print(\"The results are not the same!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Emulation of stitched IP\n",
    "\n",
    "Here we use the same procedure. First the child model is loaded, but in contrast to the layer-by-layer emulation, the metadata property `exec_mode` is set to \"rtlsim\" for the whole child model. When the model is integrated and executed in the last step, the verilog files of the stitched IP of the child model are used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from finn.transformation.fpgadataflow.insert_dwc import InsertDWC\n",
    "from finn.transformation.fpgadataflow.insert_fifo import InsertFIFO\n",
    "from finn.transformation.fpgadataflow.create_stitched_ip import CreateStitchedIP\n",
    "\n",
    "child_model = ModelWrapper(build_dir + \"/tfc_w1_a1_dataflow_child.onnx\")\n",
    "child_model = child_model.transform(InsertDWC())\n",
    "\n",
    "# set all impl_styles of the DWCs to hls to enable emulation\n",
    "dwc_nodes = child_model.get_nodes_by_op_type(\"StreamingDataWidthConverter_Batch\")\n",
    "for dwc in dwc_nodes:\n",
    "    dwc_inst = getCustomOp(dwc)\n",
    "    dwc_inst.set_nodeattr(\"impl_style\", \"hls\")\n",
    "    \n",
    "child_model = child_model.transform(InsertFIFO(create_shallow_fifos=True))\n",
    "child_model.save(build_dir + \"/test.onnx\");\n",
    "child_model = child_model.transform(GiveUniqueNodeNames())\n",
    "child_model = child_model.transform(PrepareIP(test_fpga_part, target_clk_ns))\n",
    "child_model = child_model.transform(HLSSynthIP())\n",
    "child_model = child_model.transform(CreateStitchedIP(test_fpga_part, target_clk_ns))\n",
    "child_model = child_model.transform(PrepareRTLSim())\n",
    "child_model.set_metadata_prop(\"exec_mode\",\"rtlsim\")\n",
    "child_model.save(build_dir + \"/tfc_w1_a1_dataflow_child.onnx\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# parent model\n",
    "model_for_rtlsim = ModelWrapper(build_dir + \"/tfc_w1_a1_dataflow_parent.onnx\")\n",
    "# reference child model\n",
    "sdp_node = getCustomOp(model_for_rtlsim.graph.node[1])\n",
    "sdp_node.set_nodeattr(\"model\", build_dir + \"/tfc_w1_a1_dataflow_child.onnx\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_dict = oxe.execute_onnx(model_for_rtlsim, input_dict)\n",
    "output_rtlsim = output_dict[list(output_dict.keys())[0]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if np.isclose(output_rtlsim, np.where(output_golden[0]==np.amax(output_golden[0])), atol=1e-3).all():\n",
    "    print(\"Results are the same!\")\n",
    "else:\n",
    "    print(\"The results are not the same!\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
